Closes #3

Just wanted to leave my 2cents here:
I tried your whisper changes locally and it is working perfectly on my 1080ti and Docker.
VRam is assigned and the container works as well. Home assistant also recognised and used it perfectly.
Nice one!

(Did not try Piper)

Piper does not work because of this: rhasspy/rhasspy3#49

Whisper is still targeting 20.04 is there a reason for that?

This may need to be its own image since the majority of users would not want the cuda version

could this be split into 2 tickets one for whisper and one for piper. The whisper portion is in reality the more useful of the two and benefits more from this feature. If piper is experiencing issues.

@wdunn001 From the documentation https://github.com/guillaumekln/faster-whisper/ it says it requires cuDNN 8 for CUDA 11, and for those versions of CUDA and cuDNN the highest version of ubuntu available is 20.04, and I had to look for it because it was not working with the image I set for the other containers sadly.
And updating to CUDA 12 is not planned in the very short term. See an explanation here: SYSTRAN/faster-whisper#47 (comment).

Sorry, editing because I missunderstood your comment.
Yes, makes sense to make it 2 different images, I can add that.

But I guess for better maintainability the solution we add for one should be the same as for the others, for that is I think is better to have the conversation in a single issue and PR.
If you need to use it right now you can just add the changes to your local Dockerfile and build it.
Or if you need to use CUDA 12 you could try the workarounds that they comment in here: SYSTRAN/faster-whisper#153 (comment)

And I'll try to add porcupine1 too

Awesome! I am happy to help if you need anything. Would we want to add the docker arguments for the CUDA image to the documentation here?

I added the changes.
I have not tested the new porcupine1 container, since that software does not support my language yet.

And yes, ofc we should document this, also I was thinking should we add a docker-compose.yml file?
It made sense for me since I use home assistant and need the 3 services. But now that porcupine1 has been added I am not sure anymore since as far as I know porcupine1 and openwakeword do the same, which is quite confusing for me.

But in the README.md file right now there is just the documentation for using it pulling the images, not building them, so that will depend on the tags the maintainer might wanna use. Should we add building instructions to the README.md file?

I think so for sure we can create a contributors section. I'll work on it I will be building it for the first time this weekend so I'll try and document the process.

I will give you the docker-compose files and a starting point.

I just added it, tell me how it works for you, you can create your own docker-compose.x.yml file for your use case.

I have not added porcupine1 to the docker compose because it uses the same port as openwakeword, so for that particular case it could be added in the custom extend file.

ok so I am getting an error deploying this via compose or run

usage: main.py [-h] --model {tiny,tiny-int8,base,base-int8,small,small-int8,medium,medium-int8} --uri URI --data-dir DATA_DIR [--download-dir DOWNLOAD_DIR] [--device DEVICE] [--language LANGUAGE] [--compute-type COMPUTE_TYPE] [--beam-size BEAM_SIZE] [--debug]
main.py: error: the following arguments are required: --model, --uri, --data-dir
/run.sh: line 3: --uri: command not found
/run.sh: line 4: --data-dir: command not found
/run.sh: line 5: --download-dir: command not found

It needs additional params in contrast with the other build.

These appear to be supplied by the run.sh file and I see its called in the Dockerfile.

I added commands to the GPU compose file identical to those in the NOGPU version and they work fine and made a pr. Its only the ones in the run.sh that seem to not work.

I am on Ubuntu 22.04 with latest docker is that matters.

This is weird, according to the documentation, the only thinks not extended should be volumes_from and depends_on. We can follow this discussion in the PR that you created edurenye#1

I needed to add --device cuda to actually load the whisper model onto my GPU. I second that we could split this into different branches to handle GPU for whisper, piper and wakeword. I made a branch for that, not sure if I should raise this as a PR.

• removed --cuda for piper as that isn't working upstream yet.
• changed the default data directories to /var/data to be consistent with some other docker compose files I saw.

New to contributing, happy to hear thoughts.

https://github.com/AnkushMalaker/wyoming-addons/tree/gpu

I rebased with the last chnages from master and the typos in the readme file.

I don´t think we need to create another branch for the meanwhile you can just have an extend file where you use GPU options for whisper and openwakeword and nongpu for piper.

And regarding /var/data, I am generally against storing user data in a system folder. And passing all the folder to the docker container might load a lot of data that is not needed from other applications.

@edurenye agreed using cpu for piper seems to be more than sufficient. I am still experiencing issues with openwakeword but it may just be my environment. I'll pull down the changes here and try again. I'll push any fixes I find to the PR on your branch.

I have tried applying the contents of this PR to my local instance. I do not see the faster-whisper implementation use GPU over CPU.

I have conflated the dockerfiles as such and focused on only using GPU for whisper container:

  whisper:
    container_name: whisper
    build:
      context: /opt/wyoming-addons/whisper/
      dockerfile: GPU.Dockerfile
    # image: rhasspy/wyoming-whisper:latest
    restart: unless-stopped
    ports:
      - 10300:10300
    volumes:
      - /opt/homeassistant/whisper:/data
    command: 
      - --model
      - medium-int8
      - --language
      - en
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

I can tell my GPU is passed through because it appears in nvidia-smi on the container

However when watching GPU when processing my speech the usage does not increase, and when watching CPU the usage clearly spikes since it's the CPU processing my speech

How have you all tested that this implementation of faster-whisper is working? I would like to do the same on my machine

Edit:

Found the issue!

You are missing --device in your compose

command: 
      - --model
      - small
      - --language
      - en
      - --device
      - cuda

Good finiding! Was not documented, but that parameter exists in https://github.com/rhasspy/wyoming-faster-whisper/blob/master/wyoming_faster_whisper/__main__.py

Can u resolve the conflicts? I would love to see the improvements from using the GPU directly :)

Doesn't work with piper since wyoming-piper doesn't declare the --cuda argument. I created a PR

I think removing --extra-index-url this would mean losing support for Raspberry Pi

Is that because of arm wheels? Does it even make sense to use the GPU variant of the docker container on a Raspberry Pi? You would have to use an eGPU, right?

I bet there is a way to use the --extra-index-url and get things to work. The problem is very likely a tflite issue. In general, Tensorflow is a dependency mess. It doesn't surprise me that they aren't compatible with the latest numpy. I bet there is a set of versions that can be pinned to which would fix the issue and provide Raspberry Pi support.

But right now, the Dockerfile of this code base is used for both the builds used by Home Assistant Add-ons meant to be used on Raspberry Pi and also the GPU.
I could try to fix it the same way I did with the base image, but I think there has to be a better solution that using uv instead of pip3.

And I do not understand really how using uv is fixing the problem, seems that the problem is not with this repo at all, seems more a problem with https://github.com/rhasspy/wyoming-openwakeword/blob/master/requirements.txt requiring numpy<2. And this should be fixed in rhasspy/wyoming-openwakeword#27 (comment) since the upstream openwakeword no longer requires numpy<2.

The piper (with GPU) is not working for me. (OP: ubuntu 22.04, docker standalone) What am I messing up?

INFO:wyoming_piper.download:Downloaded /data/hu_HU-imre-medium.onnx.json (https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/hu/hu_HU/imre/medium/hu_HU-imre-medium.onnx.json)

INFO:wyoming_piper.download:Downloaded /data/hu_HU-imre-medium.onnx (https://huggingface.co/rhasspy/piper-voices/resolve/v1.0.0/hu/hu_HU/imre/medium/hu_HU-imre-medium.onnx)

INFO:__main__:Ready

ERROR:asyncio:Task exception was never retrieved

future: <Task finished name='wyoming event handler' coro=<AsyncEventHandler.run() done, defined at /opt/venv/lib/python3.11/site-packages/wyoming/server.py:31> exception=FileNotFoundError(2, 'No such file or directory')>

Traceback (most recent call last):

  File "/opt/venv/lib/python3.11/site-packages/wyoming/server.py", line 41, in run

    if not (await self.handle_event(event)):

            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/opt/venv/lib/python3.11/site-packages/wyoming_piper/handler.py", line 53, in handle_event

    raise err

  File "/opt/venv/lib/python3.11/site-packages/wyoming_piper/handler.py", line 48, in handle_event

    return await self._handle_event(event)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/opt/venv/lib/python3.11/site-packages/wyoming_piper/handler.py", line 108, in _handle_event

    wav_file: wave.Wave_read = wave.open(output_path, "rb")

                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/lib/python3.11/wave.py", line 630, in open

    return Wave_read(f)

           ^^^^^^^^^^^^

  File "/usr/lib/python3.11/wave.py", line 280, in __init__

    f = builtins.open(f, 'rb')

        ^^^^^^^^^^^^^^^^^^^^^^

FileNotFoundError: [Errno 2] No such file or directory: ''

@sarpba Piper with GPU is broken, see rhasspy/wyoming#9 and rhasspy/wyoming-piper#5

how using uv is fixing the problem,

Because of the exclude-newer. It prevents newer versions of tflite from being pulled. There is probably a way to fix it by adjusting requirements to get compatible versions, but my logic was that if it worked on 2023-12-12 when the last commit was, then restricting to packages only available at that time might do the trick.

Also, more generally I would just recommend using uv because it will improve build times. The good parts of uv outweigh the bad, so much that its probably worth migrating to.

Ok, but that is not related to this issue, so it would need to be a debate and PR on its own.

I can temporally fix it so all of us can use it until rhasspy/wyoming-openwakeword#27 (comment), rhasspy/wyoming#9 and rhasspy/wyoming-piper#5 get fixed.

Because I do not think this issue can be merged until all those other issues get fixed and merged.

The diff you provided is ancient, it brings back the version 1.8.2, while we are now at the 1.10.0.

And uv would not make a difference there regarding using pip, since that was the oldest published in pypi.org, but the newer versions seem like they are being installed from GitHub instead.

Nonetheless, going back to 1.8.2 works, so I did that. If you need any feature from 1.10.0 you need to use the non GPU image.

I just have tested that the image builds and runs, not that it actually works from Home Assistant since I use Snowboy since it seems to work better for me.